Papers with shared tasks
Commonsense Inference in Natural Language Processing (COIN) - Shared Task Report (D19-60)
Copied to clipboard
| Challenge: | The workshop on Commonsense Inference in NLP (COIN) evaluated text understanding systems' ability to draw inferences about facts that are not mentioned in the text, but that are assumed to be common ground. |
| Approach: | They propose to use commonsense knowledge to evaluate systems' ability to answer questions/queries about a text. |
| Outcome: | The proposed tasks evaluated systems in two contexts: Commonsense Inference and Commonsensible Inference. |
Building a TOCFL Learner Corpus for Chinese Grammatical Error Diagnosis (L18-1)
Copied to clipboard
| Challenge: | Annotated learner corpus is valuable for research in second language acquisition, foreign language teaching, and contrastive interlanguage analysis. |
| Approach: | They construct a TOCFL learner corpus and use it for Chinese grammatical error diagnosis. |
| Outcome: | The constructed corpus is available to the public and will be used for shared tasks on Chinese grammatical error diagnosis. |
Cross-lingual Named Entity Corpus for Slavic Languages (2024.lrec-main)
Copied to clipboard
| Challenge: | This work presents a corpus manually annotated with named entities for six Slavic languages . |
| Approach: | They propose to manually annotate a corpus of names for six Slavic languages . they use a transformer-based neural network architecture to train multilingual models . |
| Outcome: | The corpus consists of 5,017 documents on seven topics . each entity is described by a category, a lemma, and a unique cross-lingual identifier. |
HiNER: A large Hindi Named Entity Recognition Dataset (2022.lrec-1)
Copied to clipboard
Rudra Murthy, Pallab Bhattacharjee, Rahul Sharnagat, Jyotsana Khatri, Diptesh Kanojia, Pushpak Bhattacharyya
| Challenge: | Named Entity Recognition (NER) is a lowerlevel task that aims to provide class labels like Person, Location, Organisation, Time, and Number to words in free text. |
| Approach: | They propose to use a standard-abiding Hindi NER dataset to analyze the annotations of a class of naming entities in free text. |
| Outcome: | The proposed dataset achieves a weighted F1 score of 88.78 with all the tags and 92.22 when we collapse the tag-set. |
Bias Analysis and Mitigation in the Evaluation of Authorship Verification (P19-1)
Copied to clipboard
| Challenge: | a paper on authorship verification shows that the underlying experiment design cannot guarantee pushing forward the state of the art. |
| Approach: | They propose a "Basic and Fairly Flawed" authorship verifier that is on a par with the best approaches submitted so far . they pinpoint sources of bias that should be eliminated and propose 'refined' authorship corpus as effective countermeasure. |
| Outcome: | The proposed approach is on par with the best approaches submitted so far . the proposed approach shows that sources of bias should be eliminated . |